A robust approach based on conditional value-at-risk measure to statistical learning problems

Robust optimization is one of typical approaches to optimize a system with incomplete information and considerable uncertainty. The standard robust optimization problem minimizes maximum cost by focusing on the considerable worst case. In some application field, it is certainly important to consider the worst case among all considerable cases, but this min-max criterion tends to lead an overly conservative decision. In this paper, we regard statistical learning problems as uncertain problems, and introduce a risk measure known as the conditional value-at-risk (CVaR) in order to dissolve overly conservativeness of robust optimization and depresses influence of outliers or measurement error which may be included in assumed uncertainty set. Monte Carlo sampling is applied to obtain an optimal solution of CVaR robust problem approximately, and convergence property of the solution is proved by using Vapnik and Chervonenkis theory. We point out that in the context of machine learning, CVaR robust problem is identical to ν-support vector classification or νsupport vector regression with apt uncertainty, and show that proposed approach is useful to deal with measurement errors in observations.